计算机愿景领域正在快速发展,特别是在神经结构设计的新方法的背景下。这些模型有助于(1)气候危机 - 增加二氧化碳排放量和(2)隐私危机 - 数据泄漏问题。为了解决经常忽视的影响计算机愿景(CV)社区对这些危机,我们概述了一个新颖的道德框架,\ Textit {P4ai}:AI的原则,是AI内伦理困境的增强原则看法。然后,我们建议使用P4AI向社区制定具体的建议,以减轻气候和隐私危机。
translated by 谷歌翻译
气候变化仍然是一个迫在眉睫的问题,目前影响社会大。重要的是,我们作为一个社会,包括计算机愿景(CV)社区采取措施限制对环境的影响。在本文中,我们(a)分析了CV方法递减递减的效果,(b)提出了一种\ entyit {'nofade''}:一种基于新的基于熵的度量来量化模型 - 数据集 - 复杂性关系。我们表明一些简历的任务正在达到饱和度,而其他CV任务几乎完全饱和。在这种光中,Nofade允许CV社区在类似的基础上比较模型和数据集,建立不良平台。
translated by 谷歌翻译
未配对的图像到图像转换的目标是产生反映目标域样式的输出图像,同时保持输入源图像的不相关内容不变。但是,由于缺乏对现有方法的内容变化的关注,来自源图像的语义信息遭受翻译期间的降级。在论文中,为了解决这个问题,我们介绍了一种新颖的方法,全局和局部对齐网络(GLA-NET)。全局对齐网络旨在将输入图像从源域传输到目标域。要有效地这样做,我们通过使用MLP-MILLER基于MATY编码器将多元高斯分布的参数(均值和标准偏差)作为样式特征学习。要更准确地传输样式,我们在编码器中使用自适应实例归一化层,具有目标多功能高斯分布的参数作为输入。我们还采用正常化和可能性损失,以进一步降低领域差距并产生高质量的产出。另外,我们介绍了局部对准网络,该网络采用预磨平的自我监督模型来通过新颖的局部对准丢失来产生注意图,确保翻译网络专注于相关像素。在五个公共数据集上进行的广泛实验表明,我们的方法有效地产生比现有方法更锐利和更现实的图像。我们的代码可在https://github.com/ygjwd12345/glanet获得。
translated by 谷歌翻译
在自动驾驶中,学习可以适应各种环境条件的分割模型至关重要。特别是,具有严重的照明变化的复制是一种推动的需求,因为在日光数据上培训的模型将在夜间训练。在本文中,我们研究了域自适应夜间语义分割(DANS)的问题,旨在学习具有标有日间数据集和未标记的数据集的判别夜间模型,包括粗略对齐的日夜图像对。为此,我们提出了一种新的双向混合(Bi-Mix)框架,用于疏浚,这可以有助于图像平移和分割适应过程。具体地,在图像翻译阶段中,Bi-Mix利用日夜图像对的知识来提高夜间图像致密的质量。另一方面,在分段适应阶段,双混合有效地桥接白天和夜间域之间的分布差距,以使模型适应夜间域。在这两个过程中,双混合简单地通过混合两个样本而无需额外的超参数来操作,因此易于实施。暗苏黎世和夜间驾驶数据集的广泛实验展示了所提出的双组合的优势,并表明我们的方法在丹盘中获得最先进的表现。我们的代码可在https://github.com/ygjwd12345/bimix上获得。
translated by 谷歌翻译
卷积神经网络在寻址像素级预测任务中的主要进展,例如语义分割,深度估计,表面正常预测等,从他们的强大功能中受益于视觉表现学习。通常,本领域模型的状态集成了对改进的深度特征表示的关注机制。最近,一些作品已经证明了学习的重要性,并结合了深度特征细化的空间和通道介绍。在本文中,WEAIM在有效地提升之前的方法和提出统一的深度框架,以便以原则的方式共同学习空间注意图和信道注意矢量,以便构建由此两种类型的注意力之间的引起的张量和模型相互作用。具体地,我们将估计和相互作用集成了概率表示学习框架内的关注,导致变分结构注意网络(Vista-net)。我们在神经网络内实现推理规则,从而允许概率的端到端学习和CNN前端参数。正如我们对六个大型数据集的大量实证评估所证明的致密视觉预测,Vista-Net在多个连续和离散预测任务中优于最先进的,从而确认在联合结构空间中提出的方法的益处 - 深度代表学习的关注估计。该代码可在https://github.com/ygjwd12345/vista-ner上获得。
translated by 谷歌翻译
Different people speak with diverse personalized speaking styles. Although existing one-shot talking head methods have made significant progress in lip sync, natural facial expressions, and stable head motions, they still cannot generate diverse speaking styles in the final talking head videos. To tackle this problem, we propose a one-shot style-controllable talking face generation framework. In a nutshell, we aim to attain a speaking style from an arbitrary reference speaking video and then drive the one-shot portrait to speak with the reference speaking style and another piece of audio. Specifically, we first develop a style encoder to extract dynamic facial motion patterns of a style reference video and then encode them into a style code. Afterward, we introduce a style-controllable decoder to synthesize stylized facial animations from the speech content and style code. In order to integrate the reference speaking style into generated videos, we design a style-aware adaptive transformer, which enables the encoded style code to adjust the weights of the feed-forward layers accordingly. Thanks to the style-aware adaptation mechanism, the reference speaking style can be better embedded into synthesized videos during decoding. Extensive experiments demonstrate that our method is capable of generating talking head videos with diverse speaking styles from only one portrait image and an audio clip while achieving authentic visual effects. Project Page: https://github.com/FuxiVirtualHuman/styletalk.
translated by 谷歌翻译
Unmanned aerial vehicle (UAV) swarms are considered as a promising technique for next-generation communication networks due to their flexibility, mobility, low cost, and the ability to collaboratively and autonomously provide services. Distributed learning (DL) enables UAV swarms to intelligently provide communication services, multi-directional remote surveillance, and target tracking. In this survey, we first introduce several popular DL algorithms such as federated learning (FL), multi-agent Reinforcement Learning (MARL), distributed inference, and split learning, and present a comprehensive overview of their applications for UAV swarms, such as trajectory design, power control, wireless resource allocation, user assignment, perception, and satellite communications. Then, we present several state-of-the-art applications of UAV swarms in wireless communication systems, such us reconfigurable intelligent surface (RIS), virtual reality (VR), semantic communications, and discuss the problems and challenges that DL-enabled UAV swarms can solve in these applications. Finally, we describe open problems of using DL in UAV swarms and future research directions of DL enabled UAV swarms. In summary, this survey provides a comprehensive survey of various DL applications for UAV swarms in extensive scenarios.
translated by 谷歌翻译
In this work, we focus on instance-level open vocabulary segmentation, intending to expand a segmenter for instance-wise novel categories without mask annotations. We investigate a simple yet effective framework with the help of image captions, focusing on exploiting thousands of object nouns in captions to discover instances of novel classes. Rather than adopting pretrained caption models or using massive caption datasets with complex pipelines, we propose an end-to-end solution from two aspects: caption grounding and caption generation. In particular, we devise a joint Caption Grounding and Generation (CGG) framework based on a Mask Transformer baseline. The framework has a novel grounding loss that performs explicit and implicit multi-modal feature alignments. We further design a lightweight caption generation head to allow for additional caption supervision. We find that grounding and generation complement each other, significantly enhancing the segmentation performance for novel categories. We conduct extensive experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS). The results demonstrate the superiority of our CGG framework over previous OVIS methods, achieving a large improvement of 6.8% mAP on novel classes without extra caption data. Our method also achieves over 15% PQ improvements for novel classes on the OSPS benchmark under various settings.
translated by 谷歌翻译
Non-line-of-sight (NLOS) imaging aims to reconstruct the three-dimensional hidden scenes from the data measured in the line-of-sight, which uses photon time-of-flight information encoded in light after multiple diffuse reflections. The under-sampled scanning data can facilitate fast imaging. However, the resulting reconstruction problem becomes a serious ill-posed inverse problem, the solution of which is of high possibility to be degraded due to noises and distortions. In this paper, we propose two novel NLOS reconstruction models based on curvature regularization, i.e., the object-domain curvature regularization model and the dual (i.e., signal and object)-domain curvature regularization model. Fast numerical optimization algorithms are developed relying on the alternating direction method of multipliers (ADMM) with the backtracking stepsize rule, which are further accelerated by GPU implementation. We evaluate the proposed algorithms on both synthetic and real datasets, which achieve state-of-the-art performance, especially in the compressed sensing setting. All our codes and data are available at https://github.com/Duanlab123/CurvNLOS.
translated by 谷歌翻译
Stance detection refers to the task of extracting the standpoint (Favor, Against or Neither) towards a target in given texts. Such research gains increasing attention with the proliferation of social media contents. The conventional framework of handling stance detection is converting it into text classification tasks. Deep learning models have already replaced rule-based models and traditional machine learning models in solving such problems. Current deep neural networks are facing two main challenges which are insufficient labeled data and information in social media posts and the unexplainable nature of deep learning models. A new pre-trained language model chatGPT was launched on Nov 30, 2022. For the stance detection tasks, our experiments show that ChatGPT can achieve SOTA or similar performance for commonly used datasets including SemEval-2016 and P-Stance. At the same time, ChatGPT can provide explanation for its own prediction, which is beyond the capability of any existing model. The explanations for the cases it cannot provide classification results are especially useful. ChatGPT has the potential to be the best AI model for stance detection tasks in NLP, or at least change the research paradigm of this field. ChatGPT also opens up the possibility of building explanatory AI for stance detection.
translated by 谷歌翻译